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Abstract 


E-marketplaces  can  be  beneficial  when  used  for  the  procurement  of  maintenance,  repair 
and  operating  (MRO)  supplies.  Our  study  identifies  both  effectiveness-  and  efficiency- 
related  attributes  of  e-marketplaces  for  procurement.  We  then  examine  these  attributes 
using  an  actual  e-marketplace  for  MRO  items  in  a  military  setting.  In  our  research,  we 
surveyed  users  of  the  NATO  logistics  stock  exchange  (NLSE)  to  obtain  perception  rat¬ 
ings  of  both  the  importance  and  performance  by  the  NLSE  on  each  of  these  attributes. 
The  resulting  classification  of  e-marketplace  attributes  both  indicates  which  attributes  are 
highly  valued  by  users,  as  well  as  identifies  on  which  attributes  an  organisation  such  as 
the  NLSE  should  focus  its  efforts  for  higher  user  satisfaction.  Efficiency-related  attributes 
appear  to  be  most  valued  by  NLSE  users. 
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Introduction 

Studies  on  e-procurement  and  e-marketplaces  are  mainly  focussed  on  the  procurement  of  raw 
materials  and  production  goods  (Harink,  2003).  Research  on  e-procurement  and  e-marketplaces 
rarely  addresses  the  procurement  of  maintenance,  repair  and  operating  (MRO)  supplies  (Croom, 
2000;  Eng,  2004).  A  considerable  share  of  companies’  procurement  activities,  however,  is 
allocated  to  the  procurement  of  MRO  items.  Typically,  MRO  items  account  for  80%  of  the 
procurement  department  personnel  workload  while  representing  only  20%  of  the  organisation’s 
purchasing  dollars  (Van  Weele,  2005).  Due  to  the  low  contribution  to  total  purchasing  turnover, 
MRO  procurement  usually  receives  little  management  attention.  As  a  consequence,  MRO  items 
tend  to  be  procured  in  an  uncoordinated  and  decentralized  manner,  barely  providing  added  value 
to  the  company  (Croom,  2000).  Moreover,  Van  Weele  (2005)  argues  that  MRO  procurement  is 
complex  and  problematic  (due  to  the  large  assortment  of  relatively  cheap  items,  many  suppliers, 
and  irregular  demand).  Several  authors  see  the  potential  of  e-marketplaces  for  reorganizing  the 
MRO  supply  chain  (e.g.,  Puschmann  &  Alt,  2005;  Shevchenko  &  Shevchenko,  2005). 

Despite  considerable  investment  and  high  expectations  of  savings  from  e-procurement,  however, 
evidence  of  such  benefits  is  still  minimal  (Howard,  Vidgen  &  Powell,  2006).  In  a  number  of 
cases,  the  realised  benefits  are  absent  or  differ  significantly  from  the  expected  benefits 
(Dubelaar,  Sohal  &  Savic,  2005).  Subramaniam  and  Shaw  (2004)  report  that  users  of  different 
forms  of  e-procurement  are  unsure  of  several  factors:  the  value  provided,  the  factors  that  affect 
the  value,  and  how  to  measure  this  value.  Croom  and  Johnston’s  study  (2003)  could  find  little 
measurement  and  evaluation  of  e-commerce  performance.  According  to  Murtaza,  Gupta  and 
Carroll  (2004),  many  e-marketplaces  have  failed  to  deliver  their  promised  benefits.  Other  authors 
agree  that  one  procurement  tool  cannot  meet  all  business  requirements,  and  that  a  portfolio 
approach  is  required  for  exploiting  the  potential  of  different  e-procurement  tools  and  e- 
marketplaces  (Bartezzaghi  &  Ronchi,  2004;  Kim  &  Shunk,  2004;  Knudsen,  2003).  To  survive,  e- 
marketplaces  should  not  only  offer  “core  services,”  but  also  “value  added”  services — such  as 
credit,  financing  and  tax  activities,  transportation  and  storage,  inventory  management  and  reverse 
logistics  (Daniel,  Hoxmeier,  White  &  Smart,  2004;  Kathawala,  Abdou  &  von  Franck,  2004; 
Shevchenko  &  Shevchenko,  2005).  As  transaction  characteristics  and  user  needs  for  non¬ 
production  goods  are  varied,  the  challenge  imposed  on  e-marketplaces  to  successfully  manage 
MRO  procurement  is  even  greater  (Subramaniam  &  Shaw,  2004).  Therefore,  further  research  on 
the  contribution  of  e-marketplaces  to  the  procurement  of  MRO  items  is  warranted. 

This  study  aims  at  evaluating  the  positive  effects  of  e-marketplaces  on  MRO  procurement  from  a 
defense  logistics  user  perspective — more  specifically,  it  focuses  on  the  NATO  Logistics  Stock 
Exchange  (NLSE).  The  NLSE  is  a  web-based,  virtual  business  place  for  co-operation  in 
logistics;  it  links  together  the  armed  forces  and  defence  industries  of  NATO  member  states  that 
wish  to  participate.  The  materials  that  can  be  exchanged  through  the  NLSE  are  defined  as,  “all 
active  and  inactive,  consumable  and  repairable  items  that  have  been  codified  with  a  NATO  Stock 
Number  (NSN),  excluding:  food,  clothing,  medical  items,  ammunition,  fuel,  crypto  and  live 
animals”,  (p.  #).  From  lifecycle  perspective,  the  NLSE  users  focus  on  the  replenishment  (or 
resupply)  phase  and  disposal  phase  of  fielded  systems,  not  on  initial  supply  for  new  military 
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systems.  According  to  the  definition  of  Kathawala  et  al.  (2002),  the  NLSE  can  be  characterized 
as  both  an  “aggregator,”  where  prices  are  pre -negotiated,  and  as  a  “matcher,”  where  buyers  and 
sellers  negotiate  prices  real-time.  The  NLSE  is  a  web-based  IT  system  that  is  managed  by  the 
NATO  Maintenance  and  Supply  Agency  (NAMSA)  in  Capellen,  Luxembourg — making  it  a  third 
party  e-marketplace.  Established  in  1958,  the  NAMSA  is  NATO’s  principal  logistics  support 
management  agency.  The  NAMSA’s  main  task  is  to  assist  NATO  nations  by  organizing  common 
procurement  and  supply  of  spare  parts  and  by  arranging  maintenance  and  repair  services 
necessary  for  the  support  of  various  weapon  systems  in  their  inventories.  This  assistance  is 
available  whenever  two  or  more  nations  operate  the  same  system  and  have  made  a  conscious 
decision  to  use  the  NAMSA’s  support  facilities.  The  NAMSA’s  main  role  is  in  consolidating 
nations’  requirements,  centralizing  logistics  management  activities,  conducting  international 
competitive  bidding  processes  and  controlling  the  cost  and  quality  of  the  services  rendered  to 
customers.  The  NAMSA’s  aim  is  to  ensure  that  customers  receive  the  very  best  logistics  support 
available  to  guarantee  the  operational  readiness  of  their  various  weapon  and  equipment  systems. 
The  NAMSA  can  be  accessed  easily  by  users  through  a  secured  Internet  connection  at 
https://www.natolog.com. 

Based  on  the  applicable  literature,  our  study  proceeds  as  follows.  We  first  identify  the  different 
attributes  of  e-marketplaces  that  contribute  to  effective  and  efficient  MRO  procurement.  We  then 
report  the  results  of  a  survey  of  64  users  of  the  NLSE  regarding  these  attributes.  By  applying  an 
importance-performance  analysis  (IP A)  and  revised  importance  analysis  (Matzler,  Sauerwein,  & 
Heischmidt,  2003),  we  gain  important  insights  into  how  the  different  attributes  are  valued.  This 
study  aims  at  identifying  key  benefits  (in  terms  of  user  value)  of  e-marketplaces  in  order  for 
resources  to  be  effectively  targeted  to  deliver  these  benefits.  It  also  formulates  implications  for 
the  NLSE  as  well  as  further  research  directions. 

Literature  Review 

Procurement  of  MRO  Supplies 

Supplies  that  are  consumed  by  a  firm  rather  than  used  to  form  part  of  its  output  are  commonly 
known  as  MRO  supplies,  indirect  goods  or  non-production  goods.  Common  problems  associated 
with  MRO  items  are:  disproportional  workload,  an  extensive  range  of  items  and  suppliers,  many 
company-specific  items  (such  as  spare  parts),  and  a  low  and  irregular  demand  for  items  (Le 
Sueur  &  Dale,  1998;  Van  Weele,  2005).  According  to  Subramaniam  and  Shaw  (2004),  MRO 
procurement  suffers  from  inefficient  buying,  redundant  and  disconnected  processes  and 
maverick  buying  (i.e.,  purchasing  outside  of  the  official  procurement  processes).  Puschmann  and 
Alt  (2005)  state  that  paper-prone  and  labour-intensive  processes  for  MRO  procurement  cause 
large  inefficiencies  and  considerable  error  potential.  A  study  about  Greek  government 
purchasing  (Panayiotou,  Gayialis  &  Tatsiopoulos,  2004)  reveals  significant  delays  in  the  MRO 
procurement  cycle  due  to  mistakes  in  procurement  requisitions,  unnecessary  and  iterative  steps 
in  the  procurement  process,  a  large  number  of  controls,  checking  and  authorisation,  and  same- 
data  entry  into  different  applications.  Moreover,  the  use  of  large  numbers  of  (copies  of) 
documents  and  the  lack  of  standardisation  leads  to  poor  dissemination  of  information  across 
companies.  Le  Sueur  and  Dale  (1998)  report  mismatching  invoices  in  the  procurement  system. 
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Kim  and  Shunk  (2004)  mention  that  procurement  of  indirect  goods  has  received  far  less  attention 
from  organisations  than  that  of  direct  goods,  resulting  in  little  procurement  process 
standardisation  for  indirect  goods.  MRO  procurement  is  associated  with  a  significant  waste  of 
money,  time  and  personnel  capacity,  a  non-availability  of  supplies  when  required,  the  holding  of 
excessive  stocks,  and  unsatisfied  internal  customers. 

Electronic  Marketplaces 

The  rise  of  the  Internet  and  the  rapid  spread  of  electronic  business  across  world  markets  have  left 
few  industries  unchanged  (Howard  et  al.,  2006).  Internet  technology  enables  enterprises  to  create 
marketplaces  in  which  electronic  transactions  between  buyers  and  sellers  can  take  place  (Eng, 
2004).  These  marketplaces  are  called  electronic  marketplaces  or  e-marketplaces.  According  to 
Daniel  et  al.  (2004),  e-marketplaces  enable  automated  transactions,  trading  or  collaboration 
between  business  partners.  E-marketplaces  can  be  categorised:  by  trading  mechanism,  by 
ownership  (buyer,  seller,  third-party),  by  number  of  owners  (one  vs.  a  consortium  of  owners),  by 
business  model,  by  trading  entities  (private  or  public),  by  type  of  goods  sold  (direct  or  indirect) 
or  by  industry  orientation  (horizontal,  vertical  or  diagonal).  Dubelaar  et  al.  (2005)  add  customer 
environment  (B2B  vs.  B2C)  as  an  extra  categorization.  As  mentioned  above,  Kathawala  et  al. 
(2002)  distinguish  aggregators,  in  which  prices  are  pre-negotiated,  from  matchers,  in  which 
buyers  and  sellers  negotiate  prices  in  real-time.  Murtaza  et  al.  (2004)  classify  e-marketplaces 
based  on  the  type  of  supplied  goods  (direct  vs.  indirect  goods)  and  buyer/supplier  relationship 
(one-time  vs.  long-term).  This  model  is  depicted  in  Figure  1.  “E-distributors”  should  be  applied 
when  a  consumer  is  spot-sourcing  for  bottleneck  products,  while  e-procurement  should  be 
applied  for  the  systems  contracting  strategy  for  routine  products. 


Supply 

Indirect  Goods  Direct  Goods 


One- 

E-Distributors 

Exchanges 

time 

Matching, 

Matching, 

Relationship 

Dynamic  pricing 

Dynamic  pricing 

Long- 

E-procurement 

Consortia 

term 

Aggregation, 

Aggregation, 

Fixed/negotiated 

Fixed/negotiated 

pricing 

pricing 

Figure  1.  Classification  of  E-marketplaces 

(Murtaza,  2004) 
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Effective  and  Efficient  E-procurement 

In  general,  purchasing  performance  is  considered  to  be  the  result  of  two  elements:  purchasing 
effectiveness  and  purchasing  efficiency  (Bartezzaghi  &  Ronchi  2004;  Van  Weele,  2005). 
Purchasing  for  military  organisations  can  especially  be  viewed  in  terms  of  these  two  metrics. 

Table  1  lists  the  expected  benefits  of  e-marketplace  usage  on  purchasing  effectiveness  and 
efficiency. 


Table  1.  Expected  Benefits  of  E-marketplace  Usage  on  MRO  Procurement 


Purchasing  effectiveness 

Purchasing  efficiency 

Increase  in  product  quality 

Reduction  of  product  search  time 

Reduction  of  product  prices 

Reduction  of  negotiating  time 

Improvement  in  delivery  perfonnance 

Reduction  in  procurement  process  costs 

Improvement  in  internal  customer  satisfaction 

Reduction  of  order  processing  time 

Reduction  of  stock  levels 

Reduction  of  payment  processing  time 

Reduction  of  excessive  stocks 

Reduction  of  maverick  buying 

Reduction  of  number  of  stock-outs 

Reduction  of  number  of  authorisation 

stages 

Reduction  of  procurement  process  failures 

Reduction  of  paperwork 

Increase  of  negotiating  power 

Improvement  in  information  exchange  with 

partners  in  the  supply  chain 

Increase  in  demand  forecasting  accuracy 

Increase  in  number  of  potential  suppliers 

Reduction  of  switching  costs  between  suppliers 

Increase  in  product  selection 

Increase  in  synchronisation  of  activities  and 

responsibilities  between  procurement  personnel  and 

internal  customers 
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Effectiveness  refers  to  the  degree  to  which  a  previously  established  goal  has  been  met  (Van 
Weele,  2005).  Purchasing  effectiveness  is  often  evaluated  by  the  quality  of  purchased  products 
and  services,  the  costs  associated  with  the  purchase  and  stocks,  and  customer  satisfaction. 
Purchasing  effectiveness  is  further  enhanced  by  the  degree  of  purchasing  clout,  the  options 
available  to  the  purchaser  and  the  ease  of  switching  suppliers. 

Efficiency  refers  to  the  relationship  between  planned  and  actual  sacrifices  made  in  order  to 
realize  a  previously  established  goal  (Van  Weele,  2005).  Purchasing  efficiency  typically  includes 
the  time  involved  and  the  number  of  steps  required  to  complete  an  order.  E-marketplaces  will 
likely  be  seen  as  beneficial  when  they  improve  the  effectiveness  as  well  as  the  efficiency  of  the 
procurement  process. 

Impact  on  Purchasing  Effectiveness 

Effectiveness  of  the  procurement  process  measures  the  direct  outcome  of  the  procurement 
process:  to  what  extent  the  procured  item  is  of  the  required  quality,  is  affordable  and  is  delivered 
at  the  right  time  and  place.  Procurement  process  effectiveness  involves  the  quality  of  purchasing 
decisions  (Kalakota  &  Robinson,  2001).  Bartezzaghi  &  Rone  hi  (2004)  conclude  that  one 
significant  objective  (when  organisations  are  aiming  for  procurement  process  effectiveness  for 
non-critical  materials)  is  improvement  in  product  quality.  Howard  et  al.  (2006)  develop  a 
framework  for  assessing  the  dissonance  between  expected  and  realised  benefits  of  e- 
marketplaces,  and  recognise  product  quality  as  a  motivation  for  e-marketplace  adoption. 

According  to  Panayiotou  et  al.  (2004),  the  largest  savings  in  business-to-govemment  e- 
procurement  of  indirect  goods  stem  from  lower  product  prices  due  to  centralised  spending. 
Murtaza  et  al.  (2004)  point  to  the  aggregation  of  demand  within  the  organisation  that  can  lead  to 
lower  product  prices.  A  survey  among  92  companies  towards  main  objectives  for  e-business 
implementation  revealed  that  70%  of  the  respondents  aim  at  product  price  reduction  (Croom, 
2005). 

E-marketplace  adoption  increases  the  number  of  distribution  channels  and  provides  organisations 
the  opportunity  to  transport  items  outside  the  internal  transportation  channels.  Bartezzaghi  & 
Ronchi  (2004)  find  that,  from  a  buying  perspective,  improved  delivery  perfonnance  is  a 
significant  benefit  of  adopting  e-marketplaces.  Croom  and  Johnston  (2003)  focus  on  the 
procurement  function  as  an  internal  service  provider  for  the  employees  within  an  organisation. 
They  conclude,  based  on  a  qualitative  study  involving  97  organisations,  that  the  adoption  of  e- 
procurement  can  lead  to  increased  internal  customer  satisfaction  on  the  availability  of  goods,  the 
responsiveness  of  the  procurement  department  (speed),  flexibility  (range  of  products)  and  care 
(fast  problem  resolution). 

Considering  trends  in  the  development  of  business-to-business  e-Hubs,  Shevchenko  and 
Shevchenko  (2005)  identify  inventory  management  as  an  added  value  provided  by  e- 
marketplaces.  For  structured  procurement  at  a  large  manufacturer  of  heavy  equipment, 
Subramaniam  and  Shaw  (2004)  show  that  the  value  of  e-marketplaces  stems  mainly  from 
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inventory  savings.  Michaelides,  Ho,  Boughton  &  Kehoe  (2003)  report  that  e-marketplaces 
provide  real-time  insight  in  stock  levels  at  supply-chain  partners — enabling  lower  stock  levels  in 
the  entire  supply  chain  and,  thus,  at  the  individual  organisations’  warehouses.  One  of  the  main 
advantages  of  the  e-business  adoption  for  two  Italian  motorcycle  companies  has  been  the 
optimization  of  the  number  of  warehouse  spare  parts  (Muffatto  &  Payaro,  2004).  Eng  (2004) 
distinguishes  the  possibility  of  communicating  both  “stock-outs”  as  excessive  stock  levels  with 
partners  in  the  supply  chain  through  e-marketplaces. 

Subramaniam  and  Shaw  (2004)  further  show  that  the  added  value  of  e-marketplaces  lies  in  the 
reduction  of  transaction  errors,  especially  for  low  and  moderate  demand  levels  of  MRO  supplies. 
Muffatto  and  Payaro  (2004)  report  a  reduced  number  of  mistakes  related  to  procurement  after  e- 
business  adoption.  Croom  (2005)  argues  that  a  major  impact  of  e-business  comes  from  its  role  as 
a  mechanism  for  improved  control  of  supply  through  collaborative  demand  forecasting.  Based 
upon  extensive  literature  reviews  and  case  studies  in  the  Italian  motorcycle  industry,  Muffano 
and  Payaro  (2004)  confirm  that  the  next  step  in  the  evolutionary  path  of  e-marketplaces  is  a 
stronger  collaboration  within  the  supply  chain.  This  enables  better  demand  planning  based  upon 
analysis  of  historical  demand  data. 

Subramaniam  and  Shaw  (2004)  report  that  better  information  visibility  of  business  processes 
leads  to  more  negotiating  power  for  the  buying  organisation.  Puschmann  and  Alt  (2005)  support 
this  view,  stating  that  the  lower  transactional  burden  in  the  procurement  process  frees  time  for 
the  procurement  department  to  focus  on  negotiating — for  instance,  when  spot-sourcing  for 
bottleneck  products  or  establishing  long-term  contracts  with  suppliers  (systems  contracting)  for 
routine  products. 

Several  authors  (Murtaza  et  al.,  2004;  Kathawala  et  al.,  2002)  emphasize  e-marketplaces’ 
potential  to  provide  buyers  with  increased  access  to  products  and  vendors.  According  to  Eng 
(2004),  e-marketplaces  remove  geographical  barriers;  therefore,  buyers  can  more  easily  find 
supply  options  and  explore  new  markets.  Subramaniam  and  Shaw  (2004)  assert  that  switching 
costs  between  suppliers  are  practically  absent  through  the  use  of  e-marketplaces. 

Croom  (2000)  describes  how  the  procurement  function  can  be  fragmented  across  the 
organisation  in  large  enterprises.  Also,  a  lack  of  direct  contact  between  procurement  personnel 
and  internal  customers  often  occurs  at  such  organisations.  E-marketplaces,  by  delivering 
transparent  information  in  real-time,  can  increase  coordination  between  procurement  personnel 
and  internal  customers  and  synchronise  their  activities  and  responsibilities.  Subramaniam  and 
Shaw  (2004)  argue  that  e-marketplaces  support  complex  coordination  within  the  organisation. 
During  the  fulfilment  of  an  order,  for  instance,  procurement  personnel  frequently  need  to 
communicate  and  exchange  information  with  the  internal  customer.  A  web-based  procurement 
system  can  provide  real-time  and  effective  information  exchange  at  lower  communication  costs. 

Impact  on  Purchasing  Efficiency 

Bartezzaghi  &  Ronchi  (2004)  distinguish  procurement  process  efficiency  from  procurement 
process  effectiveness.  They  conclude  that  companies  adopting  e-marketplaces  for  procurement 
of  non-critical  materials  aim  at  increasing  procurement  process  efficiency  and  lowering 
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procurement  process  costs.  A  main  attribute  of  increased  procurement  process  efficiency  is  the 
reduction  of  product  searching  costs  and  time:  e-catalogues  should  be  defined  and  uploaded  by 
the  supplier  or  third  party  without  requiring  the  customer  to  be  highly  involved.  Puschmann  and 
Alt  (2005)  emphasize  that  products  with  a  high  degree  of  coordination  effort  with  the  supplier 
but  with  a  low  frequency  order  are  not  suitable  candidates  for  e-procurement. 

Croom  (2005)  concluded  that  an  important  objective  of  e-business  is  an  overall  reduction  of 
procurement  process  costs.  Knudsen  (2003)  assesses  the  value  of  e-marketplaces  on  “rents” 
(above  normal  rates  of  return)  and  concludes  that  the  main  value-adding  mechanism  for  e-MRO 
procurement  lies  in  the  reduction  of  transaction  costs.  Dubelaar  et  al.  (2005)  investigated 
companies  adopting  e-marketplaces  and  found  few  differences  between  expected  and  derived 
procurement  process-cost  reduction. 

Murtaza  et  al.  (2004)  focus  on  the  “facilitation”  functionality  of  e-marketplaces  for  increasing 
the  efficiency  of  the  procurement  process;  e-marketplaces  enable  automated  posting  of 
RFP/RFQ,  bidding  and  negotiating.  Subramaniam  and  Shaw  (2004)  argue  that  this  “market¬ 
making”  functionality  of  e-marketplaces  can  replace  a  number  of  human-intensive  tasks.  For 
instance,  Panayiotou  et  al.(2004)  expect  the  Greek  government  to  save  20%  on  human  resources 
utilisation  when  evaluating  tenders.  Moreover,  e-marketplaces  employing  the  aggregation 
mechanism  for  systems  contracting  (thus,  pre-negotiated,  fixed,  product  prices)  reduce  the  entire 
negotiating  process. 

Principal  actions  in  the  settlement  phase  are  order  generation,  order  placement,  order  tracking 
and  items  receiving  (Kim  &  Shunk,  2004).  Extensive  literature  is  available  on  the  contribution  of 
e-marketplaces  to  order  process  automation;  these  studies  focus  mainly  on  the  benefits  of 
reducing  manual  actions  and  removing  unnecessary,  iterative,  steps  in  the  purchase  order  process 
(Dubelaar  et  al.,  2005;  Murtaza  et  al,  2004;  Shevchenko  &  Shevchenko,  2005).  Invoice 
verification  (i.e.,  matching  the  invoice  with  the  submission  of  the  order  and  delivery  of  the 
product)  is  an  e-marketplace  effect  that  especially  contributes  to  a  lower  transactional  burden 
(Puschmann  &  Alt,  2005).  Also,  Kathawala  et  al.  (2002)  conclude  that  Grainger.com,  a 
successful  third-party  MRO  Hub  engaging  in  systematic  sourcing  with  buyers,  offers  added 
value  to  its  customers  by  providing  further  streamlining  of  the  procurement  process  through 
payment  automation.  When  deciding  on  the  adoption  of  e-marketplaces  for  the  procurement  of 
indirect  goods,  Kim  and  Shunk  (2004)  strongly  recommend  an  organisation  integrate  the  e- 
marketplace  within  its  back-office  purchasing  or  financial  system. 

A  study  in  automotive  industries  revealed  that  improved  procurement  efficiency  in  general — and 
control  over  Maverick  spending  in  particular  (i.e.,  off-process  procurement,  such  as  cash 
purchases  of  low-value  items  with  subsequent  reimbursement) — are  benefits  that  have  both  been 
realised  as  expected  for  non-production  goods  (Howard  et  al,  2006).  Croom  and  Johnston  (2003) 
found  that  employees  with  access  to  e-procurement  systems  appreciate  their  “ease  and  speed  of 
use”  and  now  refrain  from  such  maverick  purchases. 

Another  way  of  increasing  the  efficiency  of  the  MRO  procurement  process  is  by  lowering  the 
number  of  authorisation  stages.  Based  on  a  benchmark  study  of  five  companies  that  successfully 
implemented  e-marketplaces  for  indirect  procurement,  Puschmann  and  Alt  (2005)  found  the 
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reduction  of  authorisation  stages  is  an  e-marketplace  effect  that  significantly  contributes  to  more 
efficient  MRO  procurement.  Moreover,  Kim  and  Shunk  (2004)  state  that  e-procurement  systems 
can  provide  automatic  requisition  approval  based  upon  authorisation  rules.  Puschmann  and  Alt 
(2005)  show  that  the  implementation  of  e-marketplaces  has  provided  organisations  the 
opportunity  to  redesign  existing  business  processes  and  focus  on  elimination  of  paperwork. 
Howard  et  al.  (2006)  have  conducted  a  case  study  at  Ford  motor  company  demonstrating  that 
minimising  paper  transactions  is  a  benefit  both  expected  and  realised. 

In  the  empirical  study  by  Croom  (2005),  46%  of  the  respondents  expected  the  adoption  of  e- 
procurement  to  improve  the  information  flow  with  their  supply-chain  partners.  Muffato  and 
Payaro  (2004)  report  increased  knowledge  transfer  about  suppliers,  products,  product  prices  and 
inventory  levels  between  supply-chain  partners  through  e-marketplaces.  Eng  (2004)  concludes 
that  the  improved  information  flow  offered  by  e-marketplaces  contributes  to  the  cooperation 
between  supply-chain  partners. 

Methodology 

We  have  identified  a  large  number  of  effectiveness-  and  efficiency-related  benefits  of  e- 
marketplace  procurement  that  may  reduce  MRO  problems.  To  assess  both  the  importance  and 
performance  of  these  benefits,  a  survey  was  conducted  among  users  of  the  NATO  Logistics 
Stock  Exchange  (NLSE),  an  e-marketplace  focussed  on  MRO  supplies.  A  structured 
questionnaire  was  considered  appropriate,  given  the  geographical  distribution  of  NLSE  users 
(Europe,  EISA  and  Canada)  and  the  need  for  anonymity.  Users  of  the  NLSE  were  asked  to  state 
the  importance  and  performance  (from  a  procurement  perspective)  of  all  23  e-marketplace 
attributes  (as  listed  in  Table  1)  based  upon  a  5-point  Likert  scale  (Matzler,  Bailom,  Hinterhuber, 
Renzl  &  Pichler,  2004).  Prior  to  conducting  the  survey,  the  researchers  submitted  a  clear  and 
concise  formulation  of  the  questionnaire  items  to  be  checked  and  verified  by  personnel  of  the 
NAMSA  that  manage  the  NLSE.  The  collection  of  responses  took  place  in  four  iterations  from 
April  to  late  September  2006.  During  the  NLSE  User  Conference  (held  10  and  11  April  2006  at 
the  NAMSA’s  premises  in  Capellen,  Luxembourg),  a  presentation  was  given  on  this  study,  and 
the  questionnaire  was  distributed  in  hardcopy.  Between  May  and  August  2006,  the  questionnaire 
was  submitted  three  times  per  e-mail  to  all  NLSE  users  listed  in  the  NAMSA’s  database. 

In  the  main  section  of  the  questionnaire  (reproduced  in  Appendix  1),  respondents  were  asked  to 
state  their  opinion  on  the  importance  of  each  of  the  23  attributes  for  electronic  markets  in 
general,  and  the  performance  of  the  NLSE  on  that  attribute  in  particular.  Answers  could  be  given 
on  a  5-point  scale,  where:  1  =  “very  low,”  2  =  “low,”  3  =  “not  low,  not  high,”  4  =  “high,”  5  = 
“very  high.”  The  total  number  of  valid  responses  (i.e.,  from  respondents  with  actual  experience 
in  the  NLSE  from  a  procurement  perspective)  was  64  (n  =  64).  The  number  of  NLSE  users  is 
estimated  at  180  (N=  180).  The  NLSE  survey  response  rate  is  calculated  as  (n/N)  x  100%  =  36%. 
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Analysis 

The  original  importance/performance  analysis  (Martilla  &  James,  1977)  yields  insights  on  which 
quality  (product  or  service)  attributes  a  firm  should  focus  in  order  to  achieve  customer 
satisfaction.  Based  upon  customer-perceived  importance  and  performance  of  these  quality 
attributes,  a  two-dimensional  matrix  can  be  constructed  with  the  means  of  the  performance  and 
importance  dividing  the  matrix  into  four  quadrants.  Attributes  with  high  importance  and  high 
performance  represent  opportunities  for  gaining  or  sustaining  competitive  advantage.  The 
company  should  “Keep  Up  the  Good  Work.”  A  firm  should  immediately  pay  attention  to 
attributes  with  a  high  importance  and  a  low  performance:  “Concentrate  Here.”  Attributes  of  low 
importance  and  of  low  performance  deserve  no  extra  effort;  instead,  they  should  remain  a  “Low 
Priority.”  Low  importance  and  high  performance  are  indications  of  a  “Possible  Overkill”  and 
imply  that  resources  committed  to  these  attributes  could  be  better  employed  elsewhere. 

Implicitly,  two  assumptions  underlie  the  traditional  IPA:  (1)  attribute  performance  and  attribute 
importance  are  independent  variables,  and  (2)  the  relationship  between  attribute  performance  and 
overall  customer  satisfaction  is  linear  and  symmetric.  Recently,  however,  theoretical  and 
empirical  work  has  shown  that  both  assumptions  are  not  valid  per  se,  which  calls  into  question 
the  applicability  of  the  traditional  IPA  (Matzler  et  al.,  2004).  Therefore,  we  have  conducted  a 
revised  importance-performance  analysis  for  our  study  that  is  based  upon  the  three-factor 
customer  satisfaction  model  (Kano,  1984)  and  the  calculation  of  an  implicit  attribute  importance 
(Matzler  et  al.,  2003). 

As  the  importance  of  an  attribute  can  be  seen  as  a  function  of  the  performance,  Matzler  et  al. 
(2003)  explain  that  the  relative  importance  of  an  attribute  cannot  be  represented  correctly  by  an 
explicit,  self-stated,  importance.  In  order  to  take  the  importance-performance  relationship  into 
account,  an  implicit  importance  needs  to  be  measured  that  is  based  upon  the  correlation  between 
the  perfonnance  of  the  attribute  and  an  external  criterion,  such  as  the  overall  customer 
satisfaction.  The  stronger  this  correlation,  the  more  important  the  attribute  (Anderson  &  Mittal, 
2000).  Based  upon  the  three-factor  customer  satisfaction  model  and  the  calculation  of  an 
implicit  attribute  importance,  Matzler  et  al.  (2003)  have  proposed  the  revised  Importance  matrix, 
as  depicted  in  Figure  2. 
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Figure  2.  Revised  Importance  Matrix 
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The  three  quality  attributes  are  mapped  in  the  importance  matrix  as  follows: 

■  Basic  factors  (low  implicit  importance,  high  explicit  importance) — Customers  regard 
these  attributes  as  important  (high  explicit  importance),  but  customer  satisfaction  will  not 
be  increased  if  a  basic  factor  is  fulfilled  (low  implicit  importance). 

■  Excitement  factors  (high  implicit  importance,  low  explicit  importance) — These  attributes 
are  not  regarded  as  important  by  customers  (low  explicit  importance),  but  if  they  are 
delivered,  they  generate  customer  delight  (high  implicit  importance). 

■  Performance  factors — These  show  a  linear  relationship  between  attribute  performance 
and  overall  customer  satisfaction.  They  can  be  important  (high  implicit  importance,  high 
explicit  importance)  or  unimportant  (low  implicit  importance,  low  explicit  importance). 

Results 

The  implicit  importance  of  each  e-marketplace  attribute  is  calculated  as  Pearson’s  correlation 
coefficient,  r,  between  the  attribute  performance  values  and  the  overall  customer  satisfaction, 
while  controlling  for  the  self-stated  importance  for  this  attribute.  Table  2  shows  the  calculated 
implicit  importance  as  well  as  the  self-stated  (explicit)  importance  scores. 
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Table  2.  Explicit  (self-stated)  Importance  and  Implicit  Importance 
for  E-marketplaces’  Attributes 


Attribute 

Explicit 

import 

Implicit 

import 

Attribute 

Explicit 

import 

Implicit 

import 

1. 

Reduction  of  product 
search  time 

4,14 

0,181 

13. 

Improvement  in  delivery  perform¬ 
ance 

3,84 

0,209 

2. 

Reduction  of  negotiating 
time 

3,87 

0,071 

14. 

Improvement  in  internal  customer 
satisfaction 

4,02 

0,151 

3. 

Reduction  of  order 
processing  time 

4,27 

0,317 

15. 

Reduction  of  stock  levels 

3,87 

0,484 

4. 

Reduction  of  payment 
processing  time 

3,84 

0,365 

16. 

Reduction  of  excessive  stocks 

3,81 

0,296 

5. 

Reduction  of  Maverick 
buying 

3,69 

0,389 

17. 

Reduction  of  number  of  stock-outs 

3,84 

0,199 

6. 

Reduction  of  the  number 
of  authorisation  stages 

3,80 

0,017 

18. 

Increase  in  demand  forecasting  ac¬ 
curacy 

3,70 

0,238 

7. 

Reduction  of  paperwork 

4,09 

0,126 

19. 

Increase  in  number  of  potential 
suppliers 

3,70 

0,049 

8. 

Reduction  of  procure¬ 
ment  process  costs 

4,20 

0,165 

20. 

Increase  in  product  selection 

3,61 

0,014 

9. 

Increase  of  product 
quality 

3,82 

0,245 

21. 

Reduction  in  switching  costs  be¬ 
tween  suppliers 

3,70 

0,174 

10. 

Reduction  of  product 
prices 

4,02 

0,292 

22. 

Improvement  in  information  ex¬ 
change  with  partners  in  the  supply 
chain 

3,98 

0,172 

11. 

Increase  of  negotiating 
power 

3,69 

0,152 

23. 

Increase  of  synchronisation  of  ac¬ 
tivities  and  responsibilities  between 
procurement  personnel  and  internal 
customers 

3,60 

0,071 

12. 

Reduction  of  procure¬ 
ment  process  failures 

3,98 

0,298 

With  the  calculated  scores  for  the  implicit  attribute  importance,  an  importance-performance 
matrix  can  be  constructed  (Figure  3).  The  position  of  the  grid  is  determined  by  the  two  overall 
mean  values  for  explicit  attribute  importance  and  implicit  attribute  importance,  which  are  3,87 
and  0,20  respectively. 
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Figure  3.  Importance-performance  Analysis 

(Numbers  Correspond  to  the  Attribute  in  Table  2) 


Discussion 


The  results  of  the  importance-performance  analysis  can  be  analyzed  and  interpreted  by 
classifying  the  e-nrarketplace  attributes  and  distinguishing  between  the  impact  on  purchasing 
effectiveness  and  purchasing  efficiency,  as  shown  in  Table  3. 
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Table  3.  Classification  of  Attributes 


Attribute 

Purchasing  effec¬ 
tiveness 

Purchasing  effi¬ 
ciency 

Classification 

Reduction  of  product  search  time 

Efficiency 

Basic 

Reduction  of  paperwork 

Efficiency 

Basic 

Reduction  in  procurement  process  costs 

Efficiency 

Basic 

Improvement  in  internal  customer  satisfac¬ 
tion 

Effectiveness 

Basic 

Improvement  of  information  exchange  with 
partners  in  the  supply  chain 

Efficiency 

Basic 

Reduction  of  order  processing  time 

Efficiency 

Important 

Reduction  of  product  prices 

Effectiveness 

Important 

Reduction  of  procurement  process  failures 

Effectiveness 

Important 

Reduction  of  payment  processing  time 

Efficiency 

Excitement 

Reduction  of  maverick  buying 

Efficiency 

Excitement 

Increase  in  product  quality 

Effectiveness 

Excitement 

Improvement  in  delivery  performance 

Effectiveness 

Excitement 

Reduction  of  stock  levels 

Effectiveness 

Excitement 

Reduction  of  excessive  stocks 

Effectiveness 

Excitement 

Increase  in  demand  forecasting  accuracy 

Effectiveness 

Excitement 

Reduction  of  negotiating  time 

Efficiency 

Unimportant 

Reduction  of  authorisation  stages 

Efficiency 

Unimportant 

Increase  of  negotiating  power 

Effectiveness 

Unimportant 

Reduction  of  number  of  stock-outs 

Effectiveness 

Unimportant 

Increase  in  number  of  potential  suppliers 

Effectiveness 

Unimportant 

Increase  in  product  selection 

Effectiveness 

Unimportant 

Reduction  of  switching  costs 

Effectiveness 

Unimportant 

Increase  in  synchronisation  of  activities 

Effectiveness 

Unimportant 
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Basic  factors  (low  implicit  importance,  high  explicit  importance)  are  reduction  in  product  search 
time,  reduction  of  paperwork,  reduction  of  procurement  process  costs,  improvement  in  internal 
customer  satisfaction  and  improvement  of  information  exchange  with  partners  in  the  supply 
chain.  The  NAMSA  must  ensure  that  these  attributes  meet  the  performance  level  that  is  expected 
by  the  customer.  Most  of  the  attributes  in  the  basic-factor  segment  imply  a  positive  impact  on 
purchasing  efficiency.  Users  appear  to  view  these  attributes  as  potential  dissatisfiers.  However, 
performance  above  a  certain  threshold  in  these  areas  does  not  automatically  increase  customer 
satisfaction. 

Important  performance  factors  (high  implicit  importance,  high  explicit  importance)  are  reduction 
of  order  processing  time,  reduction  of  product  prices  and  reduction  of  procurement  process 
failures.  Merely  three  out  of  the  23  attributes  are  considered  as  an  important-performance  factor. 
A  reduction  of  process  time  and  process  failures  will  result  in  more  satisfied  users.  A  better 
performance  directly  means  a  higher  level  of  customer  satisfaction.  The  NAMSA  could, 
therefore,  focus  on  these  attributes,  since  an  increase  of  performance  on  these  attributes  will 
likely  lead  to  increased  overall  customer  satisfaction. 

Excitement  factors  (high  implicit  importance,  low  explicit  importance)  are  reduction  of  payment 
processing  time,  reduction  of  maverick  buying,  increase  in  product  quality,  improvement  in 
delivery  perfonnance,  reduction  of  stock  levels,  reduction  of  excessive  stocks  and  an  increase  in 
demand  forecasting  accuracy.  Most  of  the  excitement  factors  are  associated  with  an 
improvement  in  purchasing  effectiveness.  The  importance  of  excitement  factors  depends  on  their 
performance,  as  an  excellent  perfonnance  has  a  greater  impact  on  overall  customer  satisfaction 
than  a  poor  performance.  Obviously,  users  are  surprised  by  the  performance  of  the  NLSE  in 
these  areas.  The  NLSE  could  consider  improving  these  excitement  factors,  although  the  handling 
of  commitment  to  the  basic  and  important  perfonnance  factors  should  take  priority. 

Table  4  summarizes  the  perfonnance  factors,  their  related  attributes  and  the  recommended  focus 
for  the  NLSE: 
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Table  4.  Classification  of  E-marketplace  Attributes, 
including  Recommended  Customer  Satisfaction  Strategy 


Perfonnance 

factor 

E-marketplace  attributes 

Recommended  customer 
satisfaction  strategy  for  the 
NLSE 

Basic  fac¬ 
tors 

reduction  of  product  search  time,  reduction  of  paper¬ 
work,  reduction  of  procurement  process  costs,  im¬ 
provement  in  internal  customer  satisfaction  and  im¬ 
provement  in  information  exchange  with  partners  in 
the  supply  chain 

Primary  focus.  Continu¬ 
ously  ensure  these  attrib¬ 
utes  meet  the  expected 
perfonnance  level. 

Important 

performance 

factors 

reduction  of  order  processing  time,  reduction  of  prod¬ 
uct  prices  and  reduction  of  procurement  process  fail¬ 
ures 

Primary  focus.  Continu¬ 
ously  ensure  these  attrib¬ 
utes  meet  the  expected 
perfonnance  level. 

Excitement 

factors 

reduction  of  payment  processing  time,  reduction  of 
Maverick  buying,  increase  in  product  quality,  im¬ 
provement  in  delivery  performance,  reduction  of  stock 
levels,  reduction  of  excessive  stocks  and  increase  in 
demand  forecasting  accuracy 

Secondary  focus.  Can  lead 
to  higher  satisfaction  if  ba¬ 
sic  factors  and  important 
perfonnance  factors  are 
fulfilled. 

Unimportant 

performance 

factors 

reduction  of  negotiating  time,  reduction  of  number  of 
authorisation  stages,  increase  of  negotiating  power, 
reduction  of  number  of  stock-outs,  increase  in  number 
of  potential  suppliers,  increase  in  product  selection, 
reduction  of  switching  costs  between  suppliers  and  in¬ 
crease  in  synchronisation  of  activities  and  responsibili¬ 
ties  between  procurement  personnel  and  internal  cus¬ 
tomers 

Unimportant.  Focus  on  the 
other  attributes. 

Unimportant  perfonnance  factors  (low  implicit  importance,  low  explicit  importance)  are  reduc¬ 
tion  in  negotiating  time,  reduction  of  number  of  authorisation  stages,  increase  of  negotiating 
power,  reduction  of  number  of  stock-outs,  increase  in  number  of  potential  suppliers,  increase  in 
product  selection,  reduction  of  switching  costs  between  suppliers  and  increase  in  synchronisation 
of  activities  and  responsibilities  between  procurement  personnel  and  internal  customers.  Most  of 
the  unimportant  performance  factors,  too,  are  associated  with  an  improvement  of  purchasing  ef¬ 
fectiveness.  The  impact  of  these  factors  on  overall  customer  satisfaction  can  be  considered  mar¬ 
ginal. 
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Conclusions  and  Implications 

The  buying  and  handling  of  MRO  supplies  are  traditionally  associated  with  inefficient  and 
ineffective  procurement  processes.  E-marketplaces  could  be  beneficial  for  the  procurement  of 
MRO  supplies.  So  far,  few  studies  have  reported  on  MRO  procurement,  particularly  in  relation 
to  e-marketplaces.  Our  study  identified  23  theoretical  attributes  of  using  e-marketplaces  for 
procurement  of  MRO  items.  These  attributes  are  further  examined  from  both  the  importance  and 
performance  perspective,  using  the  NATO  Logistics  Stock  Exchange — an  actual  e-marketplace 
for  MRO  items  in  a  military  setting.  It  was  found  that  users  expect  a  minimal  performance  in 
attributes  that  are  likely  to  improve  the  purchasing  efficiency.  However,  users  classify  the 
purchasing  effectiveness  attributes  as  unimportant  factors  and,  in  other  cases,  as  excitement 
factors.  The  management  of  the  NLSE  (the  NAMSA)  could  focus  on  other  attributes  in  its 
attempt  to  improve  internal  user  satisfaction.  Users  value  a  reduction  of  order  processing  time,  a 
reduction  of  product  prices  and  a  reduction  of  process  failures. 

Electronic  marketplaces  and  MRO  procurement  are  rather  comprehensive  concepts  in  the  field  of 
supply  chain  management.  To  contribute  to  the  theoretical  knowledge  in  these  areas,  the  scope  of 
our  study  was  restricted.  It  is  recommended  that  further  research  include  other  types  of  e- 
marketplaces,  for  example:  supplier-owned  or  buyer-owned  e-marketplaces,  private  e- 
marketplaces,  non-governmental  e-marketplaces,  e-marketplaces  focusing  on  non-MRO  items, 
and  B2C  e-marketplaces. 

Distinguishing  between  “core  services”  and  “value  added  services”  provided  by  e-marketplaces 
could  be  a  useful  research  avenue  as  well.  Daniel  et  al.  (2004),  Kathawala  et  al.  (2004)  and 
Shevchenko  and  Shevchenko  (2005)  stated  that  e-marketplaces  should  also  offer  “value  added” 
services  such  as  credit,  financing  and  tax  activities,  transportation  and  storage,  inventory 
management  and  reverse  logistics  tools  in  order  to  survive. 

MRO  items  can  be  classified  as  either  non-critical  (with  a  low  supply  risk)  or  bottleneck  (with  a 
high  supply  risk).  Non-critical  items  require  efficient  processing,  product  standardization,  order 
volume  and  inventory  optimization.  On  the  other  hand,  the  procurement  of  bottleneck  items 
requires  assurance  of  supply,  vendor  control,  security  of  inventories  and  backup  plans  (Kraljic, 
1983;  Olsen  &  Ellram,  1997;  Geldennan  &  Van  Weele,  2003;  2005).  Future  studies  may  assess 
the  benefits  of  e-market  places  by  departing  from  the  distinction  between  non-critical  and 
bottleneck  items. 
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Appendix  1  Questionnaire 

We  would  like  to  know  your  opinion  on  23  attributes  of  electronic  marketplaces  in  general  and 
the  NLSE  in  particular.  For  each  attribute,  you  are  asked  to  state  your  opinion  on: 

the  importance  of  that  attribute  for  electronic  markets  in  general,  and 

the  performance  of  the  NLSE  on  that  attribute. 

The  answers  are  measured  on  a  5-point  scale,  where: 

1  =  “very  low” 

2  =  “low” 

3  =  “not  low,  not  high” 

4  =  “high” 

5  =  “very  high” 

Please  give  your  answers  by  encircling  the  answer  of  your  choice. 

All  questions  should  be  answered  from  NSLE  users’  buying  perspective. 
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Attribute 

Importance  of  this 
attribute  in  general 

Performance  of  the 
NLSE  on  this  attribute 

Reduction  of  product  search  time 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Reduction  of  negotiating  time 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Reduction  of  order  processing  time 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Reduction  of  payment  processing  time 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Reduction  of  maverick  buying 

(maverick  buying  =  employees  buy  articles  from  a  source 
other  than  the  supplier  with  whom  the  company  has  negoti¬ 
ated  a  framework  agreement) 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Reduction  of  the  number  of  authorisation  stages 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Reduction  of  paperwork 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Reduction  of  procurement  process  costs 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Increase  of  product  quality 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Reduction  of  product  prices 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Increase  of  negotiating  power 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Reduction  of  procurement  process  failures 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Improvement  in  delivery  performance 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Improvement  in  internal  customer  satisfaction 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Reduction  of  stock  levels 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Reduction  of  excessive  stocks 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Reduction  of  number  of  stock-outs 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Increase  in  demand  forecasting  accuracy 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Increase  in  number  of  potential  suppliers 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Increase  in  product  selection 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Reduction  in  switching  costs  between  suppliers 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Improvement  of  information  exchange  with  partners  in  the 
supply  chain 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

Increase  in  synchronisation  of  activities  and  responsibilities 
between  procurement  personnel  and  internal  customers 
within  the  organisation 

1  -  2  -  3  -  4  -  5 

1  -  2  -  3  -  4  -  5 

How  satisfied  are  you  on  the  NLSE  on  a  scale  from  1  to  10? 
(overall  satisfaction) 

